Experiments with Linear Prediction 
in Television 

By C. W. HARRISON 

(Manuscript received February 28, 1952) 

The correlation present in a signal makes possible the prediction of the 
future of the signal in terms of the past and present. If the method used for 
prediction makes full use of the entire pertinent past, then the error signal^— 
the difference between the actual and the predicted signal — unll be a com- 
pletely random wave of lower power than the original signal but containing 
all the information of the original. 

One method of prediction, which does not make full use of the past, hut 
which is nevertheless remarkably effective with certain signals and also 
appealing because of its relative simplicity, is linear prediction. Here the 
prediction for the next signal sample is simply the sum of previous signal 
samples each midliplied by an appropriate weighting factor. The best values 
for the weighting coefficients depend upon the statistics of the signal, bid 
once they have been determined the prediction may be done with relatively 
simple apparatus. 

This paper describes the apparatus used for some experiments on linear 
prediction of television signals, and describes the results obtained to date. 

INTRODUCTION 

Linear prediction is perhaps the most expedient elementary means of 
removing first order correlation in a television message. Before discussing 
the advantages and disadvantages of linear prediction, it might be well 
to consider what is generally meant by correlation in a television pic- 
ture and why it should be removed. 

Almost every picture that has recognizable features contains both 
linear and non-linear correlation. Each type of correlation helps in 
identifying one picture from another; however, linear prediction is only 
effective in removing linear correlation, and for this reason, future ref- 
erences to correlation will refer only to its linear properties. With tele- 
vision, a signal is obtained as the result of scanning; hence, the cor- 
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relation is evident in both space and time. Briefly, correlation is that 
relation which the "next" elemental part of the signal has with its past. 
To leave correlation in a message is to be redundant, and this effec- 
tively loads the transmission medium with a lot of excess "words" not 
necessary to the description of the picture at the receiving end. It is 
then more "efficient" to send only the infoimation necessary to identify 
the picture, and to restore the redundancy at the receiver. 

EFFICIENT TRANSMISSION 

The more efficient we are in sending pictures over a given transmission 
line, the more alarmed we become at the increasing amount of equipment 
that is required at the transmitting and receiving terminals. Certainly 
the design will be a compromise between the complexity of apparatus and 
the efficiency achieved. The ingenuity of engineers T\'in be taxed along 
these lines for years to come; however basically, the general form of 
these systems wiW be similar to that shown in Fig. 1. Although not 
always separable, four essential operations are required-namely, decor- 
relating, encoding, decoding and correlating. The transmittmg decor- 
relator and the encoder encompass the principal design problems, since 
the decoder and correlator at the receiving end perform the reverse 
operations which interpret the code and add in the redundancy that was 
removed. 

Decorrelation involves prediction, and as the predictors are more 
nearly made to predict the future of the signal, the more the output 
signal from the decorrelator resembles random noise. The essential 
picture infoimation is still present, which means that our original picture 
signal can be obtained at the receiving end without theoretical degrada- 
tion. The basic job of the encoder is to match the pictiu'e information 
out of the decorrelator to the channel over which it is to be transmitted. 
There are several encoding operations. The first concerns the rate of 
information into the encoder, and that required out of it. In the case 
of television, there are flat, highly correlated areas as well as areas 
containing more concentrated detail. This means that the information 
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FiK- 1 — Block diiigram of an efficient transmission system employing reversible 
decorrelating and encoding means. 
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rate varies when the picture is scanned at the conventional uniform scan- 
ning rate. The output of tire encoder feeds a transmission line that has 
a definite channel capacity, and if maximiun efficiency is to be obtained 
from this transmission medium, then the rate of information into it 
must be held relatively uniform at a vahie near the cliannel capacdty. 
It is the job of the encoder to take the varying rate of information from 
the decorrelator and feed it to the channel at a constant rate. At the 
receiving end, the decoder must take the constant rate of information 
and deliver it to the correlator at the variable rate as originally fed into 
the transmitter's encoder. Thus, to perform this task, a variable or 
elastic delay to rmi ahead or behind, depending on the information con- 
tent of the picture being scanned, is an important part of the encoder. 
Over a long period of time, the variable delay would average out to 
some fixed value. This variable delay must never run out, even when the 
detail is concentrated. There are instances when this condition could 
not be met, such as an extended reproduction of a snow stonn; however, 
-with, good design the system should fail "safe"— a shght degradation of 
picture ciuality. This condition can be made infrequent enough to cause 
little concern. 

The encoder design must also account for noise as well as bandwidth 
of the channel and must consider the ultimate effect of an error that 
may be introduced by noise along the transmission line. As more re- 
dundancy is removed to get at the "essence" of the picture signal, the 
more impoi'tant it is to guard this "essence," as mistakes presented to 
the receiver will propagate themselves longer in the absence of correla- 
tion. Errors can be muiimized by rugged systems of modulation such 
as PCM, where the signal-to-noise ratio of the transmission line deter- 
mines the base of the PCM system selected. In any event, the encoder 
must send the information so that the effect of errors will not appreciably 
disturb the picture. 

DECOERELATION AND LINEAR PREDICTION 

Fig. 2 illustrates, in a general way, a means of decorrelatmg the sig- 
nal, Si{t). For pm'poses of explanation, the encoder and decoder have 
been omitted, and the transmission between the receiving and sending 
terminals, idealized. The predictors, P, are identical, and base their 
prediction, Sp{t), on the signal's past history. In this way, the output 
of the computer represents the discrepancy between the actual value of 
the signal sample and the predictor's prediction. By this means we are 
sending only our mistakes' — the amount by which the next picture ele- 
ment surprises us. For example, if the computer is so designed that it 
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Fig. 2 — Deeorrelator and correlator showing reversible nature of this method 

of removing redundancy. 

bases its prediction on the "previous frame," and we are transmitting 
a "still," there will be no surprises after the first frame and consequently 
no output signal. Certainly it is redundant to send the same picture 
more than once. 

Linear prediction provides an easily instrumented means of removing 
redundancy. With linear prediction the next signal sample is simply 
the sum of the previous signal samples, each multiplied by an appropriate 
weighting factor. The best values for these weighting coefficients de- 
pend on the statistics of the signal. 

Fig. 3 is a block diagram of a deeorrelator employing linear predic- 
tion. The delayed versions of the input signal can be obtained from 
taps along the delay line. The weighting coefficients for each of the 
delayed signals are selected by loss in their respective paths as shown 
by the amplitude controls. The polarity of each signal can be determined 
by the switches. The output is simply the sum of these weighted signals. 

If we consider the signal on a continuous basis (not quantized or 
sampled), hnear predictors can be characterized as ordinary linear 
filters used to predistort the frequency spectrum of the signal. As such, 
they can be designed in the frequency or time domain. However as will 
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Fig. 3— General block diagram of deeorrelator employing linear prediction. 
Linear prediction bases its prediction on the weighted sum of previous signal 
samples. 
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be shown, it is much easier to recognize circuit configurations that 
reduce redundancy in the time domain. To this end, and for purposes 
of encoding, the signal is thought of as signal samples uniformly spaced 
at Nyquist intervals. Thus, amplitude values obtained by sampling 
a 4.0 mc picture signal at g microsecond intervals serve to specify the 
signal completely. Fig. 4 shows a small portion of a television raster where 
the signal is represented by signal values spaced at Nyquist intervals, 
T. The coordinates shown are designated "with respect to the "present 
value" of the signal, Sq,q . The positive coordinate directions are shown 
by the arrows. The past is represented by positive coordinates — ^the future 
by negative coordinates. In this way, the previous value of the signal 
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Fig. 4 — A small portion of a television raster showing geometrical location of 
signal samples with relation to the "present value" of the signal, >So,o . 

taken one Nuquist interval before Sq .0 is designated by Si ,0 - the previous 
line samples by 5o,i , etc. 

METHOnS OF LINEAR PREDICTION 

As previously stated, with linear prediction the next signal sample, 
Sp(t), is simply the sum of the previous signal samples each multiplied 
by an appropriate weighting factor. Thus, 

represents the weighted sum of all the previous signal values. The error 
signal, e, as shown in Fig. 2, is represented by the difference between the 
present vaue of the signal, So,o and the predictor's prediction. 

e = So.o ~ Sp{t) 

There are several specific types of hnear prediction that deserve fur- 
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ther explanation — namely, "previous value," "slope," "previous line," 
"planar" and "circular." 

"Previous valve" prediction is Illustrated in Fig. 5. Here the prediction 
is taken to be the signal amplitude of the preceding picture element. 
The previous amplitude of the signal, <Si ,o is subtracted from the present 
value of the signal, So.o. The error signal is given as 

e =^ 00,0 — "Ji.o 

The filter characteristic can be expressed as 



(2sin^).^(?-¥ 



F(^) = (2sin^ 



This method of prediction proves to be rather effective in reducing the 
average power for most television pictures. The expression for the filter 
characteristic given above shows that the peak amplitude can be twice 
that of the original signal. 

"Slope" prediction is illustrated by Fig. 6. "Slope" prediction is so 
called because it is equivalent to passing a straight line through the 
two previous signal values, with the assumption that this line will pass 
through the next signal value. The predicted signal is given by 

Sp = 201,0 — 02,0 

The frequency and phase characteristic is expressed as 

Fioi) = [4 sin= a^ry^'""'^ 

For tliis method of prediction, the peak amphtude of the error signal 
can be as much as four times the peak amplitude of the original signal. 




Fig. 5 — Example of "previous value" prediction, where the error signal is the 
difference between the actual value of the signal and the previous value. 
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It is of interest to mention that "slope" prediction is equivalent to 
two "previous value" predictors in tandem. Three or more "previous 
value" predictors in tandem are equi\'^alent to a binomial weighting 
of the previous values of the signal to fonn a predicted signal. 

For example, the prediction for three "previous value" predictors in 
tandem is given by 

Sp = 3»Si,o — 3o2,o + 03,0 
For four "previous value" predictors in tandem 

Sp = 4»Si,o — 600,0 -h 4o3,o — 04,0 

As can be seen from the above equations, further extension of "previous 
value" tandem operation results iu a heavier weighting of picture ele- 
ments further and further from the pouit to be predicted. For most 
pictures this leads to greater errors. 

"Previous line" prediction, shown in Fig. 7, would be expected to be 
similar to previous value prediction, since a picture would presumably 
have approximately the same correlation vertically as it does horizon- 
tally. This would be the case except that our interlaced scanning system 
makes the previous line signal some 28 per cent further away from So,q 
than the closest horizontal sample, Si_,o ■ The error signal, e, is given by 
where T is a line time. The error output has a maximum peak amplitude 
of twice the input. 

fi = oOiO — '5(1,1 

The filter characteristic can be exjjressed as 
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Fig. 6 — Example of "elope" prediction. Here the next sigoal value is assumed 
to lie on a straight line that intersects the two previous signal values. 
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"Planar" piedietioii, showni in Fig. 8, is effectively tandem operation 
of "pre\ious value" and "previous line" prediction. Planar prediction 
may also be thought of as the value represented by a plane above the 
present value of the signal when passed through three adjacent signal 
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Fig. 7 — Example of "previous line" prediction. Here the error signal is the 
difference between tlie actual value of the signal and the value of the signal on 
the line directly above. 
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Fig. S— An example of ■ planar" prediction. Here the prediction is represented 
by a plane that has been passed through three adjacent signal values. 
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values, namely Si,o , So.i and Si.i . The predicted signal is given by 

Sp = ^1,0 + *So,i — iSi.i . 
The filter characteristic is given by 

F(&)) =4sin — sm-— e^ ^ ^ 

The peak error ampUtude for "Planar" prediction can be four times 
that of the input signal. 

"Planar" prediction has several good characteristics. For example, 
if Si,i and Si.o were white and .So.i black, then 5o,o would be predicted 
to be black. Thus a change horizontally from white to black would 
produce no errors. Similary, if »Si,i and jSo,! were white and ,Si,o black, 
then >So ,0 would be predicted to be black. This indicates that a change 
vertically from white to black would be predicted correctly. In this 
manner, all vertical and horizontal contours in a picture are deleted. This 
philosophy can be extended to include other directions as well. 

"Circular" prediction, illustrated in Fig. 9, is an extension of planar, 
since it deletes horizontal, vertical and 52° contours as well. A total of 
190.5 microseconds of delay is required, making the required equipment 
more elaborate. Also, as more delay is required, more noise is added. 
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Fig. 9 — Past signal samples required for "circular prediction" — a type of 
prediction which removes horizontal, vertical, and ±52° straight line picture 
contours. 
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Therefore, indefinite extension of this straight Hne contour deleting 
pliilosophy is not a paying means of prediction, at least not at the present 
state of the art of wide band delay lines. Furthermore, the increasing 
diameter of the circle for extension of circular prediction would decrease 
its accuracy for finely concentrated detail. 

Fig. 10 shows the relati\'e position of picture elements nearest iSo.o 
if a mde band field delay were available. The methods of prediction dis- 
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Fig. 10 — Small poi-tion of television raster showing signal eamplea, including 
those of previous field, which would enable time extrapolation-space interpolation 
as ii motnod of prediction. 

cussed have been essentially an extrapolation in space; however, mth a 
field delay, interpolation in space, and extrapolation in time would also 
be possible. 

EXPERIMENTAL CIRCUITRY 

Experimentally, those types of predictors that involve only a few 
Nyquist intervals of delay are easiest to mechanize. Fig. 11 shows a 
simplified schematic of a decorrelator that enables an evaluation of 
linear prediction schemes having error signals given by e = 00,0150.0 ± 
ai.o-Si.o ± 03,052,0 . This enables an evaluation of "previous value" and 
"slope" prediction. The signal is fed into a terminated delay line having 
taps at Xyquist intervals. Each of these signals is individually attenuated 
by the potentiometers in the cathode circuit of the cathode followers. 
Each output is then fed to its respective polarity switch. The D.P.D.T. 
switch determines to which side of the differential amphfier, Vi , the 
particular signal is sent. Since more than one signal may require the 
same polarity, the signals ai-e combined through "L" type resistance 
attenuators to prevent interaction between signals. The D.P.D.T. 
switches are so arranged that the other signals are unaffected when a 
polarity switch is reversed. The differential ampUfier, Vi , is a cathode 
coupled circuit lia\'ing the advantage of two identical grids which pro- 
duce opposing effects in the output. The output is then matched to the 
line by the cathode follower, V& . In this way we can transmit (1) the 
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original picture signal with either polarity and any amplitude, (2) the 
picture signal delayed by one or two Nyquist intervals with either 
polarity and any ampHtude or (3) any linear combination of (1) or (2). 

Fig. 12 is a block diagram of the experimental set-up used to investi- 
gate prediction methods that involve previous value and previous hne 
samples such as "planar," etc. The input is fed into a manually variable 
delay having 0.1 Nyquist interval steps. This delay line acts like a 
vernier for the 63.5 microsecond line delay. Effectively, it enables the 
previous line samples to be positioned directly above the previous value 
samples. 

The 63.5 microsecond delay is a so-called "acoustic" or "ultrasonic" 
delay line and was developed by Mr. H. J. McSkimin. The associated 
circuitry was developed by Mr. A. L. Hopper.* Storage is accomplished 
by a fused silica bar with quartz transducers operating at a carrier 
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Fig. 11 — Simplified schematic for "previous value" and "slope" prediction. 



• A. L. Hopper, "Storing Video Information," Electronics, 24, pp. 122-125, June, 
1951. 
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Fig. 12 — Block diagram of exj)erimKi\t.al apparatus used to investigate methods 
of prediction involving com>)inations of previous signal values along a line with 
those on the line directly above. 

frequency of 54.0 mc. The over-all video bandwidth is fiat (± 0.1 db) to 
5.0 mc. Nonlineiir distortion iy approximately one percent when the 
peak-to-peak signal to r.m.s. noise is 58 db. To give an idea of its com- 
plexity, two such luiits with their associated power supplies require a 
seven-foot relay rack for housing. The signal at Ji represents the input 
picture signal, even though it may be delayed by a small fraction of a 
Nyquist interval from the actual input signal. The signal at Ji is the 
same signal as found at Ji but delayed by one line time. Each of these 
signals is fed into terminated delay lines to enable additional signal 
samples to be obtained. The geometrical location of these signal sam- 
ples is illustrated in the upper right section of Fig. 12. Here, six signals 
are obtained instead of three as were required for "previous value" and. 
"slope" prechction. These signals are weighted and polarized in the same 
manner as the tliree signals shown in Fig. 11. The output is the sum. of 
these weighted signals and is given by 

e = ao.oiSo.o + fti.o'-S'i.o + a^.fiSi.Q + 00,150,1 + o.i.\S\,i + (12.1^2.1 ■ 

The coefficients may assume positive or negative values. 

MEASUREMENTS 

It is obvious that if we are able to predict the value of most signal 
samples closely (which we will be al>le to do if there is a large amoimt of 
correlation in the picture), then the average amplitude of our mistakes 
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vnW be much less than the average amplitude of the orighial signal. 
Thus, by using the decorrelator alone, we can send a message over a 
channel with the same bandmdth as before but with less average power. 
At first, this might sound like a worthwhile sa\'ing; however this lower 
average power is accompanied by an even higher peak amphtude which 
makes any direct saving less attractive. Furthermore, the low frequency 
attentuation of the decorrelator makes the signal vulnerable to low 
frequency disturbances, since the correlator must restore (emphasize) 
these low frequency components. 

A proper but not entirely adequate method of evaluating the effective- 
ness of a predictor is by measm'ing the ratio of signal power to error 
power. This is called "Power Reduction" and is generally expressed in 
db. Power reduction simply provides a scale by which we can weigh a 
linear predictor's capabilities. The "not entirely adequate" refers to 
the fact that minimum error power may not provide simultaneously the 
lowest amount of redundancy for that given type of prediction. 

As an example, Fig. 13 shows the power reduction for the relative 
weighting of the previous horizontal signal sample as compared to the 
present value of the signal, for three pictures-later to be described as 
Scene A, B and C. The top-most curve is for Scene B, which is a 
simple, soft picture that contains very httle detail. For this picture, 
the minimum error power coincides (within measurable limits) mth the 
minimum redundancy. For Scene A and particularly Scene C, mini- 
mum error power is considerably different than that for minimum re- 
dundancy. This difference between minimum error power and miiumum 
redundancy also apphes to decorrelators using other types of predictors 
as well. Minimum redundancy may also be a misleading criterion of a 
predictor's performance, smce the value of the prediction must depend 
on the particular type of encoder used, and some types of encoding will 
require certain types of redundant, information to be retained. 

The following pictures are representations of the error signal as photo- 
graphed from a 10-inch laboratory monitor. The signals were band 
limited to 4.3 mc. Fig. 14 represents the "original" for three scenes called 
A, B and C. These pictures represent, to a first approximation, the 
gamut of pictures normally expected to be transmitted. They are by 
no means the best or the worst pictures than can be imagined ; how- 
ever any system should be able to reproduce these pictures without 
appreciable distortion. For example, Scene C should be capable of 
being sent continuously without the elastic delay running out, etc. 

Fig. 15 shows how the error signal appears for "previous value" pre- 
diction. "Previous value" prediction is excellent for flat white or dark 
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Fig. 13 — Power reduction for various weighting coefficiente for the previous 
horizontal sample. 

areas as can be observed in the background of Scene A. Where the 
separation of a white to black area is made, the error signal is large. It 
is this large error signal that inf onns the receiver of this change in bright- 
ness, and until another change occurs, the error output is again zero. This 
type of perfomiance produced the flat grey appearance of the back- 
ground. In this way, the picture represents only changes in brightness — 
a first difference type of picture. 

It may be noted that horizontal contour lines have vanished leaving 
only vertical contours which pertain to the brightness changes that 
have occurred. This effect is especially e\'ident in Scene C The 
power reductions given in the lower loft hand corner of these pictures 
are consistent -with their complexity. 

Fig. 16 shows the error signal appearance for "slope" prediction. 
When compared to the error signal for "pre\ious value" prediction 
a finer vertical granularity is obser\'ed, and this is attributed to sudden 
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Fig. 14 — ^Three pictures as photographed from the l'a(je of a kinescope. Scene 
"A" is a picture of average complexitj'. Scene "B" is a simple, rather soft picture. 
Scene "C" is a complex, Mghly detailed picture. Roughly, these pictures represent 
the gamut of pictures normally expected to be transmitted. 
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Fig. 15 — Three pictures shouiug t.hc appeiirance of the error signal when using 
"previous value" prediction. Note tlie absence of horizontal contours. 
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Fig. 16 — Three pietures showing the appearance of the error signal when using 
'slope" prediction. 
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Fig, 17 — -Three pictures showing I he appearance of the error signal when usinf 
'previous line" prediction. Note the absence of vertical contours. 
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Fig. 18~Three pictures showing Uic apiKvii:im.>e ol' Uio error signal when using 
'planar" prediction. Note the absence of horizontal and vertical contours. 
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changes in briglitiiess. In the case of "previous vakie" prediction a sudden 
change in brightness produces only one error, where for "slope," two 
errors result. Tliis, to some extent, accounts for the lesser amount of 
power reduction measured for these scenes. 

Fig. 17 shows the apjiearance of (he error signal for "previous hue" 
prediction in the three scenes. Where vertical contour hnes are pre- 
dominately left after "preWous ^'alue" prediction shown in Fig. 15, 
horizontal contours, are more prevalent now. It can be noted that the 
power reduction for "pi'evious hne" prediction is less than that for "pre- 
vious value" prediction. Tliis is due principally to the increased distance 
of the previous hne .sample from So, a. If the closest horizontal sample 
was taken at the same distance from^ the present \'alue of the signal as 
the previous line sample, then the power reduction using these signal 
values individually for prediction would be essentially the same for most 
pictures. 

Fig. 18 shows the error signal appearance for "planar" prediction. 
Here, vertical as well as horizontal contours are deleted. In Scene A 
the tree trunk has almost (completely vanished. In Scene B the picture 
has an extremely flat appearance. Scone C exhibits the lack of hori- 
zontal and vertical contours best, since only sloping contours are left. 
The power reduction figures at the lower left hand corner also show \'ahies 
foi' miniraiun error power. For most pictures, the error power can be 
reduced by a factor of oiie-half again over the planar coefhcients by 
mocUfying the weighting coefhcients. The coefficients for this modified 
planar cnse are gi\'en liy 

These (coefficients genei'ally produce an error signal with less powei- than 
the coefficients used for "planar" prediction. 

While all pictm-es contain I'cdundancy, the error signals from these 
simple linear predictors shown in Figs. 15, 16, 17 and 18 can \isuany 
be noted still to contain large amounts of redundancy. The contours of 
the models and of the various objects are reacUly identifiable. Were all 
]-ediuidancy remo\'ed, the pictm-e woidd be completely chaotic and would 
appear very much like random noise, although greater efficiency in 
tran.smission would be achieved. For richer rewards, more sophisticated 
methods of prediction will Ije required. 
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